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We study optimal approximation of stochastic processes by polynomial 
splines with free knots. The number of free knots is either a priori fixed 
or may depend on the particular trajectory. For the s-fold integrated 
Wiener process as well as for scalar diffusion processes we determine the 
asymptotic behavior of the average L p -distance to the splines spaces, as 
the (expected) number k of free knots tends to infinity. 
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1 Introduction 

Consider a stochastic process X = (X(t)) t >o with continuous paths on a proba- 
bility space (0, 21, P ). We study optimal approximation of X on the unit interval 
by polynomial splines with free knots, which has first been treated in [TP] , 

For fc£N and r 6 No we let Il r denote the set of polynomials of degree at 
most r, and we consider the space <$>k,r of polynomial splines 



3=1 

where = <o < ■ ■ ■ < tk — 1 and 7Ti, . . . ,7Tfc € n r . Furthermore, for 1 < p < oo 
and 1 < q < oo we let denote the class of measurable mappings 
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with &k,r being considered as a subset of L p [0, 1], and we define 

e k , r (X,L p ,q) =mf{(E\\X -X\\ q Lp[01] ) 1/q : X G 9T fc , r }. 

This quantity is an average L p -distance from X to the space 3>k,r- 

A natural extension of this methodology is not to work with an a priori 
chosen number of free knots, but only to control the average number of knots 
needed. This leads to the definition = UfcLi ®k,r and to the study of the 
class 0T r of measurable mappings 

X : ft -> $ r 

with $ r being considered as a subset of L p [0, 1]. For a spline approximation 
method X G 9T r we define 

C(X) = E (min{fc e N : X(-) G $ fe>r }), 

i.e., ((X) — 1 is the expected number of free knots used by X. Subject to the 
bound Q(X) < k, the minimal achievable error for approximation of X in the 
class 9tr is given by 

e% r (X,L p ,q) = inf{(E||X - X\\l p[01] ) 1/q : X £ 9t r , CW < k}. 

We shall study the asymptotics of the quantities &k,r and e| v r as k tends to 
infinity. 

The spline spaces <&k,r form nonlinear manifolds that consist of /c-term linear 
combinations of functions of the form lit i] ' 77 with < t < 1 and n G H r . 
Hence we are addressing a so-called nonlinear approximation problem. While 
nonlinear approximation is extensively studied for deterministic functions, see 
[7] for a survey, much less is known for stochastic processes, i.e., for random 
functions. Here we refer to [2j [3], where wavelet methods are analyzed, and to 
|10j . In the latter paper nonlinear approximation is related to approximation 
based on partial information, as studied in information-based complexity, and 
spline approximation with free knots is analyzed as a particular instance. 

2 Main Results 

For two sequences (dk)keN and (bk)keN of positive real numbers we write at- « bk 
if linifc^oo a,k/bk = 1, and cik > bk if liminf/^oo ak/bk > 1. Additionally, 
dfc >c bk means C\ < ak/bk < c 2 for all A; G N and some positive constants Cj. 

Fix s G No and let VF( S ) denote an s-fold integrated Wiener process. In [TU] , 
the following result was proved. 

Theorem 1. For r G No mi/i r > s, 

e*,r(^ (s U»,l) xejy r (wW,Loo,l) x JH'+Va). 
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Our first result refines and extends this theorem. Consider the stopping time 
T r , s , p = inf{i > : inf \\W^ - 7r]| Lp[0)t] > 1}, 

which yields the length of the maximal subinterval [0, T r)SiJ) ] that permits best 
approximation of from fl r with error at most one. We have < ET r , SjP < 
oo, and we put 

(3 = s + 1 /2 + 1/p 

as well as 

c - <¥ t % 1~ /3 

and 

& s , p = ( S +l/2) s + 1 / 2 -p- 1 /f./3-/5. 
Theorem 2. Let r £ No i«t/i r > s and 1 < q < oo. Then, for p = oo, 

e£ r (W« « e fc , r (wM Loo.g) « c r , Sj00 -fc^ 1 / 2 ). (1) 

Furthermore, for I < p < oo, 

b s , P ■ c r , s , p ■ fc~( s+1 /2) < efctr (wW,£ p , q) < Cr, s , P • fc-^ 1 / 2 } (2) 

and 

e^(^,L p , g )xfc-( s+1 /2). (3) 

Note that the bounds provided by {1} and ([2]) do not depend on the averaging 
parameter q. In particular, asymptotic constants cannot explode for q tending 
to infinity. Furthermore, 



for every s £ N, but 



lim b s p = 1 

p — >oo ' 



lim b s „ = 

s — >oo ' 



for every 1 < p < oo. We conjecture that the upper bound in (i) is sharp. 

We have an explicit construction of methods Xt £ 9Tfc,r that achieve the 
upper bounds in ([]]) and i.e., 

(E||wW-X fc *||^ [M] ) 1/ ^ Cr , s , p .fc-^ 1 / 2 ) ) (4) 

see (fT0|) . Moreover, these methods a.s. satisfy 

\\W^-X* k \\ Lpm ^c r ^p-k-^/V (5) 

as well, while 

\\W (S) - X k \\ Lp[0A] > & s , p • C r ^p • fc"( s +V2) (6 ) 

holds a.s. for every sequence of approximations Xk £ 9Tfc ir . Here, & S!00 = 1. 
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Our second result deals with approximation of a scalar diffusion process 
given by the stochastic differential equation 

dX(t) = a{X(t))dt + b(X(t))dW(t), t>0, 
X(Q) = x . (?) 

Here Xq € R, and W denotes a one-dimensional Wiener process. Moreover, we 
assume that the functions a, b : R — > R satisfy 

(Al) a is Lipschitz continuous, 

(A2) b is differentiable with a bounded and Lipschitz continuous derivative, 
(A3) b(x ) + 0. 

Theorem 3. Let r £ No, 1 < q < oo, and 1 < p < oo. Then 

e k<r (X, L p , q) x e% r [X, L p , q) x fc" 1 / 2 
holds for the strong solution X of equation . 

For a diffusion process X piecewise linear interpolation with free knots is fre- 
quently used in connection with adaptive step-size control. Theorem [3] provides 
a lower bound for the L p -error of any such numerical algorithm, no matter 
whether just Wiener increments or, e.g., arbitrary multiple Ito-integrals are 
used. Error estimates in [SI US] lead to refined upper bounds in Theorem [3] for 
the case 1 < p < oo, as follows. Put 



K ( Pl ,p 2 ) = {E\\boX\\ll i[0tl] 



for 1 < p\,p2 < oo. Furthermore, let B denote a Brownian bridge on [0, 1] and 
define 

n(p) = (E\\B\\l p[01] ) 1/p . 



Then 
and 



e K1 (X,L v ,p) < r,{p) ■ K(2p/(p+2),p) ■ k- 1 ' 2 
3%\(X,L p ,p) < n(p) ■ K(2p/(p + 2), 2p/(p+ 2)) • k^ 2 . 



We add that these upper bounds are achieved by numerical algorithms with 
adaptive step-size control for the Wiener increments. 

In the case p = oo it is interesting to compare the results on free-knot spline 
approximation with average k- widths of X. The latter quantities are defined by 

d k (X, L p , q) = inf (E ( mf \\X ¥>ll! p[0 ,i])) ^ 

where the infimum is taken over all linear subspaces $ C L p [0, 1] of dimension 
at most k. For X — as well as in the diffusion case we have 

4(A,L 0O , (7 )xfc-( s + 1 / 2 ), 
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see [H H31 HH [H] and [B] . Almost optimal linear subspaces are not known ex- 
plicitly, since the proof of the upper bound for dk{X, L^, q) is non-constructive. 
We add that in the case of an s-fold integrated Wiener process piecewise poly- 
nomial interpolation of at equidistant knots i/k only yields errors of order 
(In A;) 1//2 -k~( s+1 / 2 \ see [H] for results and references. Similarly, in the diffusion 
case, methods Xk £ 9V that are only based on pointwise evaluation of W and 
satisfy C{Xk) < k can at most achieve errors of order (In A;) 1 / 2 • A: -1 / 2 , see [T7] . 



3 Approximation of Deterministic Functions 

Let r £ No and 1 < p < oo be fixed. We introduce error measures, which allow 
to determine suitable free knots for spline approximation. For / £ C [0, oo[ and 
< u < v we put 

S[u.v](f) = in f \\f - ^h p [u,v]- 

Furthermore, for e > 0, we put to. £ (/) = 0, and we define 

Tj, e (f) = M{t > Tj- 1<e (f) : S [T ._ lAf)it] (f) > e} 
for j > 1. Here inf = oo, as usual. Put Ij(f) = {e > : rj )E (/) < oo}. 
Lemma 4. Let j £ N. 
(i) Ifs£l j (f) then 

(ii) The set Ij(f) is an interval, and the mapping e i— > Tj, e (/) is strictly in- 
creasing and right- continuous on Ij(f)- Furthermore, Tj >£ (f) > Tj_i iE (/) 
if e £ Ij-i(f), and lim e -Kx, t,-, £ (/) = oo. 

(Hi) If v >— > <5[ u .„](/) is strictly increasing for every u > 0, then e <— > Tj tS (f) is 
continuous on Ij(f)- 

Proof. First we show that the mapping (it, v) t— > Si UiV ](f) is continuous. Put 
Ji = [u/2,u + (v — u)/3] as well as J2 = [v — (v — u)/3, 2v\. Moreover, let 
7r Q (i) = X)i=o a * ' f° r a G K r+1 , and define a norm on R r+1 by 

IMI = IK a |Up[u+(o-tt)/3,u-(o-tt)/3]- 

If (x, y) £ Ji x J 2 and 

11/ ^ ^ a ]U p [a,a] = <*[x,i/](/) 

then 

Hall < |k Q ||i,[x,i/] < S[ u / 2 ,2v](f) + \\f\\L p [u/2,2v}- 

Hence there exists a compact set K C R r+1 such that 
$[x,y](f) = M \\f - Tr a \\ L [XtV] 

fl til 
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for every (x,y) £ J\ x J 2 . Since (x, y, a) i— » ||/ — n a \\l p [x,v] defines a continuous 
mapping on J\ x J 2 x K 1 we conclude that i— > inf ae /f ||/ — 7r Q j|i is 

continuous, too, on J\ x J 2 . 

Continuity and monotonicity of v <— > i5[ Ui1 ,](/) immediately imply (i). 

The monotonicity stated in (ii) will be verified inductively. Let < e\ < e 2 
with e 2 £ Ij(f), and suppose that r,-_i lE1 (/) < rj_i iE2 (/). Note that the latter 
holds true by definition for j ' = 1 . From (i) we get 

^-l.nC/J.Tj,^ (/)](/) > ^Tj-l.e^/),^,^ (/)](/) = £2- 

This implies Tj tEl (f) < Tj j£2 (/), and (i) excludes equality to hold here. 

Since (/) < ||/||l p [ u ,„] , the mappings e h-> Tj. £ (/) are unbounded and 

T j>e (f) > Tj-iAf) if££^-i(/)- 

For the proof of the continuity properties stated in (ii) and (iii) we also 
proceed inductively, and we use (i) and the monotonicity from (ii). Consider 
a sequence (£ n )neN in Ij(f), which converges monotonically to e £ Ij{f), and 
put t = lim tl _ >00 Tj, en (/). Assume that lim IWOO Tj-_i iEn (/) = Tj_i, e (/), which 
obviously holds true for j = 1. Continuity of (u, «) i— > and (i) imply 

tfu-.^ E m,t](/) = £) so that t < Tj_ £ (f). For a decreasing sequence (£„)„ e N we 
also have Tj >e (f) < t. For an increasing sequence (e n )neN we use the strict 
monotonicity of v t—t S\ u> v]{f) to derive £ = Tj te (f). □ 

Let F denote the class of functions / £ C [0, oo[ that satisfy 

< oo (8) 

for every j £ N and e > as well as 

limr i , s (/) = (9) 

for every j 6 N. 

Let k £ N. We now present an almost optimal spline approximation method 
of degree r with fc — 1 free knots for functions f £ F. Put 

7 fe (/)=inf{e>0:r )t , £ (/)>l} 

and note that © together with Lemma[U(ii) implies 7fc(/) £ ]0, oo[. Let 



r. 



(/)(/) 



for j = 0, . . . , k and define 



k 



fk(f) = \, 1 )r j -i,r j ] •argmin 7renr ||/-7r|| ip[Tj ._ l!T3] . (10) 
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Note that Lemma |4] guarantees 

l|/-^(/)lk[T 3 -x,r i ]=7fe(/) (11) 

for j = 1 , . . . , k and 

r fe > 1. (12) 
The spline </?£(/) I [o,i] G ^fe.r enjoys the following optimality properties. 



G 



Proposition 5. Let fceN and f G F. 
(%) For 1 < p < oo, 

II/-^(/)IIl p [o,i] <^- 7fc (/)- 
fiij For p = oo and every ip G <&k,r, 

11/ - Vlk^io.i] ^ 7fe(/)- 
(mj For 1 < p < oo, every ip G $fc !r , and every m£N wrai/i m > k, 
HZ-^lkio,!] >(m-fc)^-7 ro (/). 

Proof. For p < oo, 

11/ - ^COII^o.xi < £ II/- ^COHW^i - fc • (-»(/))* 

follows from (fTTj) and (fT2|) . For p = oo, (i) is verified analogously. 

Consider a polynomial spline ip G 4>fc !r and let = to < • • • < tk = 1 denote 
the corresponding knots. Furthermore, let p G ]0, 1[. For the proof of (ii) we 
put 

°J = T J,p-7 fc (/)(/) ■ 

for j — 0, . . . ,k. Then erfc < 1, which implies 

for some j G {1, . . . , k}. Consequently, by Lemma [4] 

11/ - ^llioo[0,l] ^ 11/ - VlUoofo-l,^] > ^ 11/ - ""IIl^^-i,^-] = P • 7fc(/)- 
For the proof of (iii) we define 

o-e = Tf> 7m (/)(/) 
for I = 0, . . . , m. Then a m < 1, which implies 

Wii-l,<Tli\ Q [tji-l,tji] 

for some indices 1 < j\ < ... < j m -k < an d 1 < £i < . . . < ^ m -fc < ?w. 
Hence, by Lemma [U 

m—k 

"/ - ^ £ * f ii/ - *u W^] = (*»-*)• ^ • (im(f)y. 
»=i 

for 1 < p < oo. Letting p tend to one completes the proof. □ 
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4 Approximation of Integrated Wiener Processes 



Let W denote a Wiener process and consider the s-fold integrated Wiener pro- 
cesses W {s) denned by W {0) = W and 

W (s) (t) = f W ( ' _1) (u)du 
Jo 

for t > and s G N. We briefly discuss some properties of W^ s \ that will be 
important in the sequel. 

The scaling property of the Wiener process implies that for every p > the 
process (p~( s+1 / 2 ) ■ W^(p ■ t))t>o is an s-fold integrated Wiener process, too. 
This fact will be called the scaling property of W^ s \ 

While W^*' has no longer independent increments for s > 1, the influence 
of the past is very explicit. For z > wc define zffl inductively by 

z W {0) (t) = W{t + z) - W{z) 

and ^ 

z W {s \t) = f Z W {S - 1] '(u) du. 
Jo 

Then it is easy to check that 

W is) (t + z) = Y L W {s - l) {z) + z W (s \t). (13) 

i=0 

Consider the filtration generated by W, which coincides with the filtration 
generated by W^ s \ and let r denote a stopping time with P (r < oo) = 1. Then 
the strong Markov property of W implies that the process 

T W& = { T W {s \t)) t > 

is an s-fold integrated Wiener process, too. Moreover, the processes T W^ and 
(1[ T ](i) ■W(t)) t >o are independent, and consequently, the processes T W^ and 
(l[o,r](i) • (t))t>o are independent as well. These facts will be called the 
strong Markov property of W^ s \ 

Fix s £ No. In the sequel we assume that r > s. For any fixed e > we 
consider the sequence of stopping times Tj^ e (W^), which turn out to be finite 
a.s. and therefore are strictly increasing, see Lemma 01 Moreover, for j G N, we 
define 

^, e =r J -, e (WW)-r i _i ie (WW). 

These random variables yield the lengths of consecutive maximal subintervals 
that permit best approximation from the space II r with error at most e. Recall 
that F C C [0, oo [ is defined via properties ([5]) and © and that (3 = s+l/2+l/p. 
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Lemma 6. The s-fold integrated Wiener process satisfies 

p(w {s) e F) = 1. 

For every e > the random variables form an i.i.d. sequence with 

&, e = e 1/fJ • £1,1 «W E(a,i)<oo. 
Proof. We claim that 

E(r ijC (W w ))<oo (14) 

for every j e N. 

For the case j = 1 let Z = S[ ^(W^) and note that 

5 m {W^) = t?-Z 
follows for t > from the scaling property of W^ s \ Hence we have 

P (n, e (W«) < t) = P («y [0jt ](W (s) ) > e) = P (Z > e ■ t-P), (15) 
which, in particular, yields 

Tl ^{W {s) ) =e 1/,3 -r 14 (H/ (s) ). (16) 

According to Corollary [T71 there exists a constant c > such that 

P (Z < rf) < exp(-c ■ jj-V^+Va)) 

holds for every 77 € ]0, 1]. We conclude that 

F(n A (W s) ) >t) <exp(-c-i) 

if t > 1, which implies E(ri,i(iy( s ))) < 00. 

Next, let j > 2, put r = r J -_i, e (W r W) and r' = 7-j, e (W w ), and assume that 
E (t) < 00. From the representation (fl~3| and the fact that r > s we derive 

iM(ff (s) )=VM^ (s) ). 

and hence it follows that 

T' =T + T hE ( T W^). (17) 

WehaveE(Ti, e ( T VF( s ))) < 00, since is an s-fold integrated Wiener process 

again, and consequently E (r') < 00. 

We turn to the properties of the sequence ^ e . Due to (jTHJ) and (fTT)) we have 

^ =riiE ( T ^))l Tli£ (^))i^. 6i , 

Furthermore, £j i£ and (l[o jT ](t) • (t))t>o are independent because of the 
strong Markov property of W^, and therefore £ J;£ and (£i, e> • • • , £j-i,e) are 
independent as well. 
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It remains to show that the trajectories of a.s. satisfy ©. By the 

properties of the sequence £j jE we have 

T he {W {s) )=e 1/p -T hl {W {s) ). (18) 
Observing (fl4|) we conclude that 

P(limT J - £ (VF (s) ) >t) = lim F {t^ £ {W (s) ) > t) 

= limP(T,-i(W w ) > t/e 1/f3 ) = 

£^0 

for every t > 0, which completes the proof. □ 

Because of Lemma [6j Proposition [5] yields sharp upper and lower bounds for 
the error of spline approximation of W^- 8 ' in terms of the random variable 

v k = lk (w (s) ). 

Remark 7. Note that a.s. satisfies \[ u>v ] £ II r for all < u < v. 

Assume that p < oo. Then v t— > 5\ U>V ](W^) is a.s. strictly increasing for all 
u > 0. We use Lemma 01 (hi) and Lemma [B] to conclude that, with probability 
one, T4 is the unique solution of 

T KVk (W^) = l. 

Consequently, we a.s. have equality in Proposition El(i) for 1 < p < oo, too. 
Note that with positive probability solutions e of the equation Tk, e {W^ 3 ') = 1 
fail to exist in the case p = oo. 

To complete the analysis of spline approximation methods we study the 
asymptotic behavior of the sequence V" fe . 

Lemma 8. For every 1 < q < oo, 

(E^) 1/9 «(fc.E(a,i))- /3 . 

Furthermore, with probability one, 

T4«(fc-E(Ci,i))^. 

Proof. Put 

k 

S* = l/*-£>.i 

3=1 

and use (fT5)) to obtain 

P (V k < e) = P (t m (W (s) ) > 1) = P (kr 13 ■ S'jT' 3 < e). (19) 



Therefore 
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and for the first statement it remains to show that 

E(S-^)«(E(£ M ))-^. 

The latter fact follows from Proposition [15l if we can verify that £1,1 has a 
proper lower tail behavior (j28[) . To this end we use (fT5|) and the large deviation 
estimate (1321) to obtain 



p(a4<^) = p(<5[o,i](^ (s) )>^) 

<P(l|W«lk [ o, 1] >rT /3 ) 
< exp(-c • ij~ 213 ) 

with some constant c > for all r\ < 1. 

In order to prove the second statement, put 

4=1 

where \i = E (£1,1) and c 2 denotes the variance of £1^. Let p > 1. Then 
P (14 > p • (fc ■ A*)"' 9 ) = F (S k < p~ 1/l3 ■ m) = P (5* < A; 1/2 • p) 

with 

p=(/>- 1/ ^-l)/a-/i<0, 

due to (fTTJj) . We apply a local version of the central limit theorem, which holds 
for i.i.d. sequences with a finite third moment, see [TH Thm. V.14], to obtain 

F(V k >p-(k-n)- ) 

< ci • k' 1/2 • (1 + k 1/2 • |p|)- 3 + (2tt)- 1/2 • / exp(-u 2 /2) du 



< c 2 ■ kr 2 

with constants c, > 0. For every p < 1 we get 

P(V k <p-(k-p)-P)<c 2 -k- 2 (20) 

in the same way. It remains to apply the Borel-Cantclli Lemma. □ 

Observe Remark [7] and use Lemma [8] to obtain (|4|) and |5} for the methods 
X* = ip* k (W^). Clearly, (@J implies the upper bounds in ©, ©, and ©. 
For the proof of the lower bound in ([2]) we apply Proposition [5j (iii) with 

m= [f3/(s + 1/2) -k\. 

Then we have 

(m-k)^-(EV^ 

« fc"( s +V2) . p-Vp . . ( s + 1/2Y+ 1 / 2 ■ (E (£1,1))^, 
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as claimed. Using the second statement in Lemma [SJ the lower bound in © is 
shown in the same way. 

It remains to prove the lower bounds for e| v r (W( s ), L p , q) in Theorem [5] 

Proof of the lower bound in ([I]) . Let k G N and consider X k G 9t r such that 
C(X k ) < k, i.e., 

oo 

^£-P(B £ )<fc (21) 
1=1 

for B t = {X(-) G \ $e-i. r }, where $ .r = 0- By Proposition [SJ(ii), 

oo 

E l^ (s) -^IIL[o,i]^E E ( 1 ^-^/)- 

For g£]0,l[, /i = E (£1,1), and L G N we define 

A^ = {^>p-(^ M )-^}, 

and 

i 

C L = (J B/. 
Since 7*(/) > -fe+i{f) for / G F, we obtain 

E ll^ (s) -^IIL[o,i] 

L oo 

>5>(i fl/ .y*) + £ E(i B ,-y/) 

L oo 

>^E(l B , n ^-V2)+ E E ( 1 B l nA l -Vf) 

£=1 t=L+l 

OO 

> P q ^- pq - (r^-P(c L ni t )+ £ ^ ? -P(^n^)). 

2=L+1 

From ([20]) we infer that P (Af) < c\ ■ l~ 2 with a constant c\ > 0. Hence there 
exists a constant C2 > such that 

oo 

T(L)=L- f3q -P(C L )+ £- /3q -F(Bz)-c 2 -L- l3q - 1 

i=L+X 

satisfies 



p-i^-E\\W^ -X k \\ q LM >T(L) (22) 



for every L G N. 
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Put a = (1 + 2/3q)/(2 + 2(3q), and take L(k) e [k a - 1, k a ] . We claim that 
there exists a constant C3 > such that 

k^ ■ T(L(k)) > (l - fc-( 1 - Q ) /3? ) 1+/3? - c 3 • fc~ 1/2 . (23) 

First, assume that P(C L ) > fc^ 1 -")/ 3 "?. Then 

fc' 3 ' 3 • T(L(k)) > k^ ■ (V Q/3 « • P(C L ) - c 2 • (k a - 

> 1 - C 3 ■ fc-V2 

with a constant C3 > 0. Next, assume P (Cl) < k~( 1 ~ aS>/3q and use (j2"Tj) to derive 

00 

X _ k -(l- a )f}q < ^ p(_g^ 
OC 

= ^ (£ • p {B t )f^ x+ ^ • (r"« • p (B|))V(i+0*) 

/ 00 v p q /(l+pq) , 00 s l/(l+/3q) 

< 53 *-p(b<)) • 53 r^-p(^) 

,00 \ l/(l+/3g) 

< fc /3?/d+w . j 53 r^-p(^)) 

Consequently, 

(00 
53 r' 39 -F(Bt) -02 • (fc Q - l)-^- 1 
e=L+i 

> (l-k-^-^y^ 9 ~ C3 .k-^, 



which completes the proof of ([23]). By (J22J) and ([23]) . 

E||wW-X fc ||^ [0il] >pV^ 9 -fc _/39 

for every p e ]0, 1[. □ 

Finally, for the proof of the lower bound in ([3]) it suffuces to establish the 
lower bound claimed for e^ r {W^ s \ Lx, 1). For further use, we shall prove a more 
general result. 

Lemma 9. For every s € N there exists a constant c > with the following 
property. For every X 6 y\ r , every A £ 21 with P (A) > 4/5, cmd every t £ ]0, 1] 
have 

E{l A -\\W^-X\\ Lim )>c-t s +^-k-^\ 
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Proof. Because of the scaling property of it suffices to study the particular 
case t = 1. Put 

B = {Xe $ 2fe , r }, 

and observe that P (B) > 1/2 follows from £(X) < k. Due to Lemma [5] and 
Proposition [5] (iii) , 

Is • ||l^ (s) -X\\ Ll {o,i] >tB-2k-V 4k - 
Put /x = E (£i,i), choose < c < (2/i) - ^, and define 

At = {V fc > c • AT"}. 

By (HU) we obtain 

P (£>*) = P (5 fc < c- 1//3 ) > P (S fe < 2/i). 

Hence 

lim P (D k ) = 1 

/c — >oo 

due to the law of large numbers, and consequently P(B(1 Dk) > 2/5 if k is 
sufficiently large, say k > fc . We conclude that 

lUnsnD 4fe • - £|k[0,l] > lAnsnD 4fc • c • 2 1 - 20 ■ fc"( s+1 / 2 ) 

and P (A (1 B n £*4fe) > 1/5 if 4fc > fco- Take expectations to complete the 
proof. □ 

Lemma [9] with A = fl and i = 1 yields the lower bound in ((3|) 

5 Approximation of Diffusion Processes 

Let X denote the solution of the stochastic differential equation ([7]) with initial 
value Xo, and recall that the drift coefficient a and the diffusion coefficient b 
are supposed to satisfy conditions (Al)-(A3). In the following we use c to 
denote unspecified positive constants, which may only depend on xq, a, b and 
the averaging parameter 1 < q < oo. 
Note that 

E ll^llL[o,i] <°° ( 24 ) 

and 

E( sup \X(t)-X( Sl )\A <c-{s 2 - Sl y/ 2 (25) 
for all 1 < q < oo and < Si < s 2 < 1, see P- 138]. 
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5.1 Upper Bounds 

In order to establish upper bounds, it suffices to consider the case of p — oo and 
r = 0, i.e., nonlinear approximation in supremum norm with piecewise constant 
splines. 

We dissect X into its martingale part 

M{t) = f b{X{s))dW(s) 
Jo 

and 

Y(t)=x + [ a(X(s))ds. 
Jo 

Lemma 10. For all 1 < q < oo and k 6 N, i/iere exists an approximation 
Y G 9t/c,o such that 

{ E \\ Y -ni M ) 1/q <c-k-\ 

Proof. Put || 5 || Lip = sup < s<t < 1 \g(t)-g(s)\/\t-s\ for g : [0,1] — > R, and define 

k 
3=1 

By (Al) and (24]), 

e ||y - P < e ||y||L p • k-< < c ■ (1 + e ||x||^ [0il] ) . < c . 

□ 

Lemma 11. For all 1 < q < oo and k e N, there exists an approximation 
M 6 yik,o such that 

(nM-M\\ q Laa[0il] y /q <c-k-v 2 . 

Proof. Let 

k 

^=J2 l Kj-i)/k,j/k]-x((j-i)/k). 

i=i 

Clearly, by 

(E\\X-X\\l [Q1] ) 1/q <c-k-^. 

Define 

R(t)= [ b(X(s))dW s . 
Jo 
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By the Burkholdcr-Davis-Gundy inequality and (A2), 

d . s a/2\ 1 /i 



(E\\M-R\\l^ [01] y /9 <c - (e(J (b(X( S ))-b(X(s))) 2 d s y /2 
<c-(E\\X-X\\l 2[01] ) 1/q 

<c-k~ x ' 2 . (26) 



Note that 

R = R + V, 

where 

k 

R = ^2 t ]ti-i)/k,j/k] ••ff(Cy'-i)A) 

3=1 

and 

k 

V = E %-DA,j7*] ' - !)/*)) ' ( W - W (U - l)/k)). 

3 = 1 

According to Theorem [2J there exists an approximation W £ Wl^o such that 

1/(2?) 



(E\\W-W\ff M 



<c-k- 1 ' 2 . 



Using W we define V G 9^2*:, o by 

k 

V = E • 6 (*(C? - !)/ fc )) ' (W - W((j - l)/k)). 

i=i 

Clearly, 

\\V - ^ll^io.i] < \\b(X)\\ Laa[0>1] W\\ Loa[0 ,xy 
Observing (|24|) and (A2), we conclude that 

/ - \ 1/5 / o \ l/(2<?) / \ 1/(29) 

(E ||V - HLp,i]) ^ ( E IIWIlSoP,!]) • ( E II w - ^112. [o,i]) 

<c-£T 1/2 . (27) 

We finally define M G 9t 2 M by M = # + V. Since 
M-M = (M -R) + (V- V), 
it remains to apply the estimates (f!?6")) and (f^T 7 )) to complete the proof. □ 
The preceding two lemma imply ek,o(X, L^, q) < c- fc -1 / 2 . 
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5.2 Lower Bounds 

For establishing lower bounds it suffices to study the case p = q = 1. Moreover, 
we assume without loss of generality that b(x ) > 0. 

Choose 77 > as well as a function 6 : R — > K such that 

(a) bo is differentiable with a bounded and Lipschitz continuous derivative, 

(b) M xeR b {x) > b(x Q )/2, 

(c) bo = b on the interval [xo — r\, xq + ?/]. 

We will use a Lamperti transform based on the space-transformation 

f x 1 
9( x ) = / FTT 



t b o{ u ) 

Note that p' = l/b and 3 " = -&£,/&§, and define H U H 2 : C[0, oo[~> C[0, oo[ by 

^1 (/)(*)= [\g'a + g"/2-b 2 )(f( S ))ds 
Jo 

and 

H 2 (f)(t) = g(f(t)). 
Put H = H 2 - Hi. Then by the Ito formula, 



*o(^W) 

The idea of the proof is as follows. We show that any good spline approxima- 
tion of X leads to a good spline approximation of H(X). However, since with a 
high probability, X stays within [xo — 77, xo + 77] for some short (but nonrandom) 
period of time, approximation of H(X) is not easier than approximation of W, 
modulo constants. 

First, we consider approximation of H\{X). 

Lemma 12. For every k 6 N there exists an approximation X\ e Vtk,o such 
that 

E\\H 1 (X)-X 1 \\ Ll[0>1] <c-k- 1 . 

Proof. Observe that \g'a + g"/2 ■ b 2 \(x) < c ■ (1 + x 2 ), and proceed as in the 
Proof of Lemma [TUJ □ 

Next, we relate approximation of X to approximation of H2{X). 

Lemma 13. For every approximation X € 9I r there exists an approximation 
X% € 9t r such that 

ax 2 )<2-ax) 

and 

E \\H 2 (X) - X 2 [k [0 ,i] < c • (E \\X - X\\ Lll0 ,i] + 1/CW)- 
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Proof. For a fixed u> G SI let X(uj) be given by 



A- 



We refine the corresponding partition to a partition = to <•••<% = 1 
that contains all the points i/£, where £ = ((X). Furthermore, we define the 
polynomials 7Tj G n r by 



A- 



Put / = X(u>) and define 



k 



X 2 {w) 



E 1 



with polynomials 



Let £ = X 2 (w). If t G jtj-i,^] C](i-l)/£,i/4 then 

|ff 3 (/)(*)-/a(*)| 

= |.9(/W) - 9(0,-1)) - g'(f(tj-i)) ■ fc(t) - 

< !<?(/(*)) - g(f(tj-i)) - g'iffc-i)) ■ (f(t) - f(tj-i))\ 

+ \g'(f(t j - 1 ))\-\f(t)-n j (t)\ 

<c(|/(t)-/(t J _ 1 )| 2 +|/(t)-7T i (t)|) 



<c- sup |/(t) - /((i - 1)/£)| 2 + - ^-(i) 



E \\H 2 (X) - X 2 \\ Ll[0A] < c • (1/C(X) +E \\X - X\\ Ll[0A] ). 



Finally, we establish a lower bound for approximation of H(X). 
Lemma 14. For every approximation X G 9T r , 




Consequently, we may invoke ((25)) to derive 



Moreover, £(X 2 ) < 2 • ((X). 



□ 



E\\H(X)-X\\ Ll[0A] >c-(C(X))- 1 / 2 . 
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Proof. Choose to £ ]0, 1] such that 



{ sup \X(t) -x \ <r) 



} 



L te[o,t ] 



satisfies P (A) > 4/5. Observe that 




with k ~ 1 < £(X) < ^7 an d choose X\ and X2 according to Lemma [T2l and 
Lemma [T3l respectively. Then 



< E \\H 2 {X) - X 2 \\ Lim + E WH^X) - X!\\ Lim 

< c • (e \\x — x\\ Lim + (C(x)r 1 + k- 1 ) 

<c-(E||X-X[[ £l[0 ,i ] +*- 1 )- 
On the other hand, C(^2 - ^1) < ((^2) + k < 3 ■ k, so that 



A Convergence of Negative Moments of Means 

Let (£j)ieN be an i.i.d. sequence of random variables such that £1 > a.s. and 
E(£i) < co. Put 



E\\H(X)-(X 2 -X 1 )\\ Ll[0A] 



follows from Lemma [Ml We conclude that 



E\\X-X\\ Lim >c-k- 1 ' 2 , 



as claimed. 



□ 



k 



s k = i/k-j2& 



Proposition 15. For every a > 0, 



liminfE(V*)>(E(£i)) 



P(6 <v) <c-v p , 



v e }0,v ] , 



(28) 



for some constants c, p, vq > 0, then 



lim E(S fe " a ) = (E(a)) 

fc — >oo 
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Proof. Put /j, = E (£1) and define 

g k (v) = a ■ -P(S k <v). 

Thanks to the weak law of large numbers, P (Sk < v) tends to l] ftoo r(u) for 
every every v p. Hence, by Lebesgue's theorem, 

poo 

lim / g k (v)dv = p' a . (29) 

Since 

poo poo 

E(S k : a )= / F{S k - a >u)du= / g k {v)dv 



the asymptotic lower bound for E (5r a ) follows from 

Given (f2"5)) , we may assume without loss of generality that c ■ Uq < 1. We 
first consider the case £1 < 1 a.s., and we put 

/W 2 rxo/k 

A k = I g k {v)dv and B k = I g k (v)dv. 

Jxo/k JO 

For vo/k < v < /x/2 we use Hoeffding's inequality to obtain 
9k(v) < a ■ v-^+V ■ P (\S k ~ Ml > M/2) < « • (k/v Q ) a+1 ■ 2 exp(-fc/2 • p 2 ), 
which implies 

lim A k = 0. 

k — >oc 

On the other hand, if pk > a, then 

f v ° / k \ 

B k = k a+1 -a- v- {a+1) ■P(y2^<v)dv 

<k a+1 -a- „-(«+!) . (p(^ < 
■/o 

< fc a+1 • a ■ c k • / v pk - < - a+1 ^ dv 
Jo 

= k a+1 ■ a ■ (pk - a)- 1 ■ c k ■ v p Q k - a , 

and therefore 

lim B k = 0. 

k — >oc 

In view of (|29p we have thus proved the proposition in the case of bounded 
variables £j. 

In the general case put f^jv = minjiV, ^} as well as S k ,N = • Si=i Ci.JV) 
and apply the result for bounded variables to obtain 

limsupE(5 fc - a ) < inf hmsu P E(5^) = inf (E^)^ = (E^y a 

k ~>oo N ^ N k^oo N ^ 

by the monotone convergence theorem. □ 
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B Small Deviations of from II, 

Let X denote a centered Gaussian random variable with values in a normed 
space (E, \\ ■ ||), and consider a finite-dimensional linear subspace II C E. We 
are interested in the small deviation behavior of 

d(X,U) = inf \\X - tt]]. 

7rGn 

Obviously, 

P (11*11 < e) < P {d(X, n) < e) (30) 

for every e > 0. We establish an upper bound for P (d(X, IT) < e) that involves 
large deviations of X, too. 

Proposition 16. //dim(II) = r then 

P (d(X, IT) < e) < (4A/e)' r • P (\\X\\ < 2e) + P (\\X\\ > A - e) 
for all A > e > 0. 

Proof. Put Bs(x) — {y G E : \\y ~ x\\ < 5} for x £ E and 5 > 0, and consider 
the sets A = II n S A (0) and £? = B e (0). Then 

{d(x,n) <e}c{x e ^i + s}n{||A:|| > A-e}, 

and therefore it suffices to prove 

P(X eA + B)< (4A/e) r -¥(\\X\\ < 2e). (31) 

Since 1/ A • A C II n i?i (0) , the e-covering number of A is not larger than 
(4A/e) r , see [2 Eqn. (1.1.10)]. Hence 



Ac yB E (ij) 

i=l 

for some x\, . . . , x n € E with n < (A\/e) r , and consequently 

n 

A + Bc \jB 2e ( Xi ). 
i=l 

Due to Anderson's inequality we have 

P(x e B 2£ ( Xl )) <F(X e B 2£ {0)), 

which implies (f3"Tj) . □ 

Now, we turn to the specific case of X = (W^ (t))te[o,i] and E = L p [0, 1], 
and we consider the subspace II = n r of polynomials of degree at most r. 
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According to the large deviation principle for the s-fold integrated Wiener 
process, 

-logP(||W^)|| Lp[04] >t)xi 2 (32) 

as t tends to infinity, see, e.g., [5], Furthermore, the small ball probabilities 
satisfy 

- logP (\\W^ \\ Lp[0il] < e) x e -V(<+i/a) (33) 
as e tends to zero, see, e.g., [TT] and [T2"] . 
Corollary 17. For all r, s G N , 

-io g p(d(^ (s) ,n r ) <£) X e -l/(«+l/2) 

as e tends to zero. 

Proof. Use (f30]> and (f33|) to obtain the upper bound. For the lower bound 
employ Proposition \W\ with A = e _1 , and note that 

-logP(||jy«|| £p[0ll] > 1/e-e) d -logPCIIWWll^p,!] < 2e) 
as £ tends to zero, due to (l32l and (l33l). □ 
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